COVID-19 pneumonia chest radiographic severity score: variability assessment among experienced and in-training radiologists and creation of a multireader composite score database for artificial intelligence algorithm development.

OBJECTIVE: The purpose was to evaluate reader variability between experienced and in-training radiologists of COVID-19 pneumonia severity on chest radiograph (CXR), and to create a multireader database suitable for AI development. METHODS: In this study, CXRs from polymerase chain reaction positive COVID-19 patients were reviewed. Six experienced cardiothoracic radiologists and two residents classified each CXR according to severity. One radiologist performed the classification twice to assess intraobserver variability. Severity classification was assessed using a 4-class system: normal (0), mild (1), moderate (2), and severe (3). A median severity score (Rad Med) for each CXR was determined for the six radiologists for development of a multireader database (XCOMS). Kendal Tau correlation and percentage of disagreement were calculated to assess variability. RESULTS: A total of 397 patients (1208 CXRs) were included (mean age, 60 years SD ± 1), 189 men). Interobserver variability between the radiologists ranges between 0.67 and 0.78. Compared to the Rad Med score, the radiologists show good correlation between 0.79-0.88. Residents show slightly lower interobserver agreement of 0.66 with each other and between 0.69 and 0.71 with experienced radiologists. Intraobserver agreement was high with a correlation coefficient of 0.77. In 220 (18%), 707 (59%), 259 (21%) and 22 (2%) CXRs there was a 0, 1, 2 or 3 class-difference. In 594 (50%) CXRs the median scores of the residents and the radiologists were similar, in 578 (48%) and 36 (3%) CXRs there was a 1 and 2 class-difference. CONCLUSION: Experienced and in-training radiologists demonstrate good inter- and intraobserver agreement in COVID-19 pneumonia severity classification. A higher percentage of disagreement was observed in moderate cases, which may affect training of AI algorithms. ADVANCES IN KNOWLEDGE: Most AI algorithms are trained on data labeled by a single expert. This study shows that for COVID-19 X-ray severity classification there is significant variability and disagreement between radiologist and between residents.

Clinical course and outcomes of COVID-19 in rheumatic disease patients: a case cohort study with a diverse population.

OBJECTIVE: To determine clinical course and outcomes in rheumatic disease patients with coronavirus disease 2019 (COVID-19) and compare results to uninfected patients. METHODS: We conducted a case cohort study of autoimmune disease patients with COVID-19 (confirmed by severe acute respiratory syndrome coronavirus 2 PCR) from February 1, 2020, to July 31, 2020, and compared them in a 1:3 ratio with uninfected patients who were matched based on race, age, sex, and comorbidity index. Patient demographics, clinical course, and outcomes were compared among these patient groups. RESULTS: A total of 70 rheumatic disease patients with COVID-19 (mean age, 56.6 years; 64% African American) were identified. The 34 (49%) patients who were hospitalized used oral glucocorticoids more frequently than those treated as outpatients (p < 0.01). All 10 patients using anti-TNFα medications were treated as outpatients (p < 0.01). Those hospitalized with COVID-19 more often required ICU admission (17 (50%) vs 27 (26%), p = 0.01) and intubation (10 (29%) vs 6 (6%), p < 0.01) than uninfected patients and had higher mortality rates (6 (18%) vs 3 (3%), p < 0.01). Of the six COVID-19 patients who died, only one was of African ancestry (p = 0.03). CONCLUSION: Rheumatic disease patients infected with COVID-19 were more likely to require ICU admission, ventilation, and died more frequently versus uninfected patients with autoimmune disease. Patients on anti-TNFα medications were hospitalized less frequently, while those on chronic glucocorticoids were hospitalized more frequently. These findings have important implications for medication choice in rheumatic disease patients during the ongoing spread of COVID-19. Key Points • We show that hospitalized rheumatic disease patients with COVID-19 have poorer outcomes including ICU admission, ventilation, and death compared to hospitalized rheumatic disease patients not infected with COVID-19. • This study adds further support regarding protective effects of anti-TNFα medications in COVID-19 disease course, with 0 of 10 of these patients required hospitalization.